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COMPARING ACTUAL AND PREDICTED DROPOUT RATES 
TO EVALUATE PROGRAM EFFECTIVENESS 



At the end of 1990» personnel in the Austin Independent Sdioot 
Distria^ (AISD) Office of Researdi and Evaluation (ORE) 
made an extensive search for reports on effective dropout pro* 
grams. Over 60 calls were made to school systems around the 
countiy to request copies of reports on effective dropout preven- 
tion practice. The net result vm thr^ studies that were based 
uponfonnalevaluationpracti<^thatdocumentedsuccess* Each 
of these three reports included some data that showed an impaa 
on the actual dropout rate of a group of students* Of the dozens 
of other reports received, the major shortcoming was the lack of 
any measure of the dropout rate. Most did measure related 
factors such as attendance, achievement^ attitude, participation 
rates, but they foiled to measure the ultimate criterion toward 
which they all were designed to work-reduction of the dropout 
rate. 

This was not only surprising but also disappointing. As much 
emphasis as is now being placed upon graduation from high 
school, we expected more school systems to have clearly defined 
methods for reporting dropout rates and for evaluating their 
prevention efforts based upon those rates. This search for 
successful practices further motivated us to pursue the method- 
ology described in this paper for the evaluation of dropout 
prevention programs. 

The ultimate criterion on which a dropout prevention program 
should be evaluated has to be the dropout or graduation rate of 
the program participants. However, practically, we often need 
interim measures that indicate program effectiveness or meas- 
ures that at least tell us whether a student's levels on character- 
istics that are typically associated with dropping out are improv- 
ing. From this perspective, we can create a hierarchy of criteria 
for evaluating dropout prevention programs. 

As we evaluate new programs, we typically rely upon the crite- 
rion of partial succ^s or criteria associated with success for 
short-term indications of impact. If the evaluation never goes 
beyond these criteria, then the evaluator has not provided an 
adequate assessment of the program's true viability. 

As we work with the criterion of partial suixess to determine if 
students stayed in school ionger than they would have without 
the program, and with the criteria associated with success to 
determine if the factors associated with dropping out are being 
improved, we need to be able to compare alternative programs 
equitably. 



CRITERIA 
FOREVALUATINC 
DROPOUT PREVENTION PROGRAMS 



Cfxitrim^Far^ Success: 



OitfHaAMMxkUdwiA Success: 



Did the ^udeflu earn 
ft high school diploma? 



Did Uie students etm 
an eqaivaknt to a hi^ 
school dipkma, or 
enter college, or 
complete training in 
a vocational program? 



Did ihe sttidents stay in 
school longer (cam more 
credits) than they would 
have without Ihc program? 



Did the studenu* at- 
risk (Mcton beocHDe 
less negative? (For 
example If attendance 
isconetatedwith 
graduating, then did 
the students* average 
daity attendance rates 
improve?) 
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HGURE 1 

H.B. lOlO: THE STATE AT-RISK CRITERIA 
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H3. 1010, pass^ by tbe Texas State 
Legislature in 1986 and taidng effect 
September 1, 19S7, retatw: to reducing the 
number of students who drc^ out of 
public schoc^ S^ion 4(f) of Ibis biQ 
states: 

Far the purposes of this section, '^student 
at risk of dropf^g out of school" includes 
each student in gr^ tevcls seven through 
12 who is under 21 years of age and who: 

( 1 ) w^ not advanced frooi one grade 
tevel to the xssi two or more 
school years; 

(2) has mathematics or reading skills 
that are two or more years below 

/ grade level; 

(3) did not maintain an average 
equEvatem to 70 on a scale of 100 
in two CH' mem courses during a 
semester, or is not maintaining 
such an average in two or more 
courses in the current semester, 
and ts not expected to graduate 
within four ^ars of the date the 
student begins the ninth grade; or 

(4) did not perfora) satisfiactorily on an 
assessment instrument admini- 
stered under Section 21 JS 1(a) of 
this code in the seventh, ninth, or 
twelfth grade. 



Grades 7->12 
19TAC75.1!»fc>fn-f4) 

Below 21 years of age and meet one or 
more of the following: 

(1) has not been prraimed one or 
more times in grades basedoo 
academic criteria established in 
subsections (a) and (b) of this 
section and continues to be unable 
to mamr tte essential element in 
the 7th or higher grade level; 

(2) is two or mxe years below grade 
level in reading or mathematics; 

(3) has failed at least two courses in 
one or nnxe semesters and is not 
expected to graduate within four 
years of the time the student 
entered the 9th grade; ch* 

(4) has failed one or more of the 
reading, writing, or mathematics 
seaions of the mo^ recent 
TEAMS test beginning with the 
seventh grade. 



Grades 7>12 
TEC21JS7m 



Under 21 years of age and who: 



(1) was not advanced from one grade 
level to the next two or more 
school years; 

(2) has tnatbematics or reading skills 
that arc two or more years below 
grade level; 

(3) did iKst maintain an average 
equivalent to 70 on a scale of 100 
in two or more courses in the 
current semester, an*J is not 
expected to graduate within four 
years of the daic the student 
begins the ninth grade; or 



(4) did not perform satisfactorily on an 
assessment instrument adminin- 
stered under Section 2l351(a> of 
this code in the seventh, ninth, or 
twelfth grade. 



ILB. lOtO ttmeacM the T«sm Edwortlm Code (TEC) guidelines wtikfi are craloloed lo tlie 
TexM Admlntotrsllve Code (TAQ. Pvovfslons In both the TEC iind TAC mmt be Implenieofed 
asUw. 

A student who meets one or more of these cHteiti shall be IdenUned as at risk A student does 
not hove to meet all four criteria to be considered ai rM(» 



Optional oilerta for identifying at-r1sl( 
studentSt grades 1^12^ are also Included i 
follows: 



Grades M2 
19 rAC7S.19Sfc) (S^ 
Optionat criteria: 



* environmental factors, 

* familial faaors» 

* economic factors, 

* social factors, 

* developmental factors, 

* other psychosocial factors where 
such factor contritnites to the 
students' inability to progress 
academically. 



Grades 7-12 
TECll^SfO 
Optionat criteria: 

^adjudged delinquent; 

* abuses drugs^cohoi; 

* limited English proficiency 

* receives compensatory or remedial 
instruction; 

^se^ataliy, physically, or psychologi- 
cally abused; 

* pregnant; 

* stow learner; 
•underachteven 
^enrolls late in school year; 

* stops attending school before the 
end of the school year, 

^unmotivated; or 

^ other characteristics that indicate 
the student is at high risk of 
dropping out* 
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Risk 
Fftctort 



Ag9 

RMd Ach 

MflthAch 
2Pi 

TEAMS Read 

TEAMS Math 
TEAMS Lang 

TEAMS WRTTE 
TEAMS WCOMP 



Aga. Read Ach 
or Math Ach 

Aga. 2 F's 



Age. TEAMS (any) 



Moth Ach or 
Raad Ach & 2 Pa 

Math Ach or 
Read Ach & 
TEAMS (any) 

TEAMS (any) 

Age. Math Ach 
Of Read Ach. 
ai2Pa 

Age, Math Ach 
or Read Ach. 
&TEAMS (any) 

Age. 2 Ps. & 
TEAMS (any) 

Age. Math Ach or 
Read Ach. 2 Ps. 
& TEAMS (any) 



Math Ach & 
Reading Ach 

TEAMS (two) 

Math Ach or 
Read Ach. 2 Ps. 
&TEAMS (any) 
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Student la two or more years otdar than expected for the grade level 

Student scored two or more years below grade level in reading on a norm-referenced, 
stendardized achievement teat (either the fowa Testa of Basle Skills or the Tests of 
Achievement and Proficiency) 

Student scored two or more years below grade level In mathematics on a nomv 
referenced, atandardbad achievement test (either the IT^ or the TAP) 

Student failed at least two courses during a semester 

Student failed the reading section en the most recant administration of the state- 
mandated. critedon-referefTced Texaa Educationat Aaaessment of Minimum Skills 
(TEAMS) (grades? and 9 only) 

Studant failed the mathenwtlce section of the TEAMS 

Student failed the language aits section of the Exit*Level TEAMS (grades 1 1 and 1 2 
only) 

Student failed the writing section of the TEAMS (Grades 7 and 9 only) 

Studant failed oniy the writing composition portion of the TEAMS Wrltinig test (grades 
7 ar>d 9 only) 

Student is tM^ or more yean ofder than expected for the grade level and scored two 
or more years below grade level In reading or mathematics on the ITBS or TAP 

Student la two or more years ofder than expected for the grade level and failed at 
least two courses during a semester 

Student Is two or more years older than expected for the grade level and failed at least 
one of the sections of the TEAMS 

Student scored two or more years below grade level in mathematics or reading on the 
fTBS or the TAP and failed at least two courses during a semester 

Student scored two cr more years below grade level in mathematics or reading on the 
fTBS or the TAP and failed at least one of the sections of the TEAMS 



Shjdent failed at least two courses during a semester and failed at least one of the 
sections of the TEAMS 

Student is two or more years older than expected for the grade level, scored two or 
more years below grade level in mathematics or reading on the PTBS or the TAP. and 
and failed at least two courses during a semester 

Student Is two or more years older than expected for the grade level, socred two or 
more years below grade twel In mathematics or reading on the ITBS or the TAP, and 
f alUtd at least one of the sections of the TEAMS 

Student Is two or more years older than expected for the grade level, failed at least 
one of the sections of the TEAMS 

Student is two or more years older than expected for the grade level, scored two or 
more years below grade level In mathematics or reading on the ITBS or the TAP, 
failed at least two courses during a semester, and failed at least one of the sections 
of the TEAMS 

Student scored two or more years below grade level In mathematics and in reading 
on the fTBS or the TAP 

Student failed at least two sections of the TEAMS. 

Student scored two or more years below grade level In mathematics or reading on the 
ITBS or the TAP. failed at least two courses during a semester, and failed at least one 
of the sections of the^AMS. 
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Afierobtatning a dropout risk probability for each student in the 
Distria in grades 742« a predicted dropout rate can be obtained 
for a group. Summing the dropout risk probability for each 
student in the group and dividing by the total N in the group 
provides agroup dropout risk probability for any defined group. 
This procedure can be extended to any number of groups one 
wishes to define or any type of group one wishes to define^ such 
as grade level or individual campuses. From this initial informa- 
tion a predicted dropout rate can be obtained for a group which 
is then compared with the actual dropout rate for that group. 
Let's look at two examples. 

EXAMPLE PROGRAM A (An X indicates student is at risk in a 
categoiy.) 



r 

StodcDti 


Age 




CRT 


NRT 


Rbk 

Gimp 


Factor 




Nsmberi 


X 








I 


30M 


IS 


Nttmb^2 


K 






X 


to 


24JI 


m 


Numbers 


X 


X 






II 


52J2 


OUT 


Nttiiiber4 


% 




X 




12 


4S.75 


OUT 


Numbers 


X 


X 


X 




it 


37.70 


IN 



MemRMiFuctor 3419 
34.19 X S $ltKleot» « 1.7 Frcdktcd Dmpoxts 
^0 ActttsI Dropouts/ L7 Predicted Ihopootfi m t.ia 



Therefore, Example Program A would be credited with a drop- 
out rate that is 1 18% of its predicted rate. 

EXAMPLE PROGRAM B 



f 

StucteDtB 


Age 




CRT 


NRT 


Risk 
Croup 


Risk 
Fuctor 


Aduut 
Stelue 


N«ml»ert 


X 




X 




12 


4SL7S 


IN 


Number 2 


X 




X 




12 


45.75 


IN 


Number 3 


X 




X 




12 


45.75 


oirr 


Naiober4 


X 




X 




12 


45.75 


OUT 


Numbers 


X 




X 




12 


45.75 


IN 



Meuo Risk Factor 45.75 
45.75 X 5 Stydenis » 23 Pmfktc^ DnH^-'Uttn 
2.0 Actual Dropotttft / 2J Pmtkted Dropoiila 0,g7 
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When Programs A and B are compared with each other, the 
evaluation would conclude that Program B is more effective* 
because it serv^ students who were at higher risk of dropping 
out and kept them in school at the same rate as did Program A. 

Obviously, because students drop out in whole numbers and 
these predicted rat^ are calculated in fractiotis of students, the 
larger a program is, the more logical this type of comparison 
becomes. In the example above. Program A has a predicted 
dropout rateof 1.7 students. Either 1.0or2,0 (or any other whole 
number up to 5.0) students could actually drop out; therefore, 
the program cannot match its predicted rate, but must be over or 
under. 

A more perplexing issue is whether within the risk grouf^ 
programs are selecting for service those students that are at 
greater risk than their group peers. For instance,d program that 
targets tbc^ students who have recently decided to dropout and 
tries to keep them in or bring them back might be scndng 
students whose true predicted dropout rate is close to 100% 
rather than what would be prediaed using the at risk group's 
dropout rat^ These "crisis intervention'* programs may be less 
appropriately evaluated using this methodology than are those 
programs that enrol! students at the beginning of a semester or 
school year. 



MODIFICATIONS 



This procedure was first used in 1988^ to evaluate the effec- 
tiveness of 1 1 dropout prevention programs serving students in 
grades 9-12 and eight dropout prevention programs serving 
students in grades Dropout risk probabilities were deter- 
mined for each student as of the end of the school yean From 
that, a predicted dropout r^te for each group was determined 
and compared to the actual (obtained) dropout rate. See 
Figure 3. 

We were left with several questions after this initial application. 
Were the dropout risk rat^ different for high school and middle 
school students? Was it legitimate to compare fall-only pro- 
grants and spring^onty programs using dropout risk prot>abiIt- 
tiescatcuiatedasoftheendofthesixthsixweeks? Furthermore, 
keeping students in school during the school year is one thing. 
We also need for them to return the following fall if they have not 
graduated. What were the dropout rates as of the end of a 
calendar year? How did the programs compare with each other 
then? 



Therefore, Example Program B would he credited with a drop- 
out rate that is 87% of its predicted rate. 
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t988S9 Dropout Ham for Selected Programs, 
PrtdU^ Obtaiaedt and <^tmned as Permtt qfPndiaed 
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GRADES 9-12 










FALL 1988 


SPRING 1989 






Predicted 


Obtained 


Obtained 




Predicted 


Obtained 


Obtained 










TP Ul 








^ f>f 

/C Lit 


Pro pram 

Jt m well MAftl 


N 


Rate 


Rate 


Predicted 


N 


Rate 


Rate 


Predicted 


CIS 


N/A 


N/A 


N/A 


N/A 


130 


6.72 


6.9 


102.7 


rVAF 


464 


14.29 


24.4 


170.7 


627 


10.15 


6.1 


60.1 


UKAL/ 


813 


13.45 


37.8 


281.0 


1.163 


10.96 


31.6 


288.3 


JLC 


51 


11.09 


12-2 


110.0 


46 


7.65 


4.3 


54.8 


Mentor 


N/A 


N/A 


N/A 


N/A 


38 


3.30 


0 


0 


PAL 










122 


7.26 


0.8 


11.0 


PEAK 


N/A 


N/A 


N/A 


N/A 


86 


8.31 


5.8 


69.8 


Rice 


86 


13.82 


31.4 


227.2 


192 


10.10 


14.1 


139.6 


Robbins 


186 


12.99 


23.7 


1814 


239 


9.93 


12.1 


121.9 


TAP 


48 


12.14 


16.7 


137.6 




# 




• 


Zenith 


28 


19.57 


42.9 


219.2 


57 


13.42 


1.8 


13.4 


V 



















Note to Readers: The authors realize these programs names and acronyms are too cryptic for most readers. These lists are 
included to be illustrative of the numbers and range of programs evaluated. 



GRADES 7-8 





FALL 1988 


SPRING 1989 






Predicted 


Obtained 


Obtained 




Predicted 


Obtained 


Obtained 






Dropout 


Dropout 


%of 




Dropout 


Dropout 


%of 


Program 


N 


Rate 


Rate 


Predicted 


N 


Rate 


Rate 


Predicted 


AIP 


219 


11.95 


8.7 


72..8 


206 


1Z07 


1.5 


12.4 


CIS 


N/A 


N/A 


N/A 


N/A 


99 


7.60 


1.0 


13.2 


CVAE 


155 


7.74 


2.6 


33.6 


124 


7.52 


0 


0 


PAL 










166 


10.59 


1.2 


11.3 1 


Rice 


111 


20.09 


24.2 


120.4 


196 


14.66 


0 


0 1 


Robbin.s 


70 


22.26 


30.0 


134.8 


45 


30.72 


35.6 


1 15.9 


TAP 


39 


11.00 


17.9 


162.7 


12 


24.05 


30.8 


128.1 


WIN 


144 


10.40 


4.2 


40.2 


78 


8.86 


11.7 


132.1 


















J 



• Number of students is too small for analysis. 
** Incomplete data. 

N/A = Not applicable. Program began service in spring, 1988. CIS service may have been yearlong, but data were not provided 
until spring 
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With these questions in mind, after that first experience* it was 
decided to undertake three modifications: 

Modification 1: ProtmbiUly by separate grade span 

Modification 2: Probability by seriester 

Modification 3: Probability by individual campus 

The first modification of the procedure was to derive the drop- 
out risk probability for grades 9-12 separate from grades 7^ 
The next modification was to have the dropout risk probabilities 
calculated differently for fall or year-long programs from spring- 
only programs. 

Fall and year-long programs now have a dropout risk probabil- 
ity calculated for the end of the fall semester, for the end of the 
school year, and for the end of the calendar year based on the fall 
enrollment Separate calculations are prepared for grades 9-12 
and grades 7-8. See Figure 4 and 5. 

Spring programs now have a dropout risk probability calculated 
for the end of the spring semester and, in order to reflect summer 
dropouts, calculated as of October. Both calculations are based 
on spring enrollment. Seprate calculations are prepared for 
grades 9-12 and grad^ 7-8. Figure 6 and 7. 

The final modification was to apply this procedure to the individ- 
ual school campuses. We knew there were difference in actual 
dropout rates between campuses. Were there differences in the 
predicted dropout rates between campuses? How welt were the 
campuses working with their assign^^ population? Were some 
campuses more effective in keeping students in school? 

When we applied the procedure to alt campuses, we discovered 
the need for a risk probability value for students who are not at 
risk. These students do not have the value 0. Giving tbc^ 
studentsa value of 0 would underpredict the number of dropouts 
for any group having even one not-at-risk student. Students who 
are not at risk do drop out. Therefore, they have the value of the 
number of students not at risk who dropped out divided by the 
number enrolled who are not at risk. This value varies by 
semester and grade level 
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RESULTS 



WiU rates dQTef ifmlmlatfd sxparaufyfor studem ingrades9-12 
andfarsasdausbt grades 7^? 

In 1988-89 there were 941 students in grades 7-12 at risk in risk 
category 1. Of those, 361 dropped ou! for a risk rate of 30.4%. 
We recalculated for grades 9*12 and for grades 7^. 



Al-RiskStudenr 
Grodes 7-12 941 
Grades 9-12 710 
Grades 7-S 231 



Dropouts % 
361 3&4 
310 43.7 
51 22*1 



As yoM can see, the recalculations display quite dramatically that 
the original calculation had given numbers which, when applied 
to programs, had the potential to overpredia greatly the numl>er 
of dropouts for middle school and greatly underpredict the 
number of dropouts for high school. 



WiU rotes d^er \f cakulated separatefyfor spring and fall? 

For l^-90we calculated for springand fall separately also. To 
simplify matters for illustrative purposes^ lefs look at the results 
of this separate calculation using the combined numbers for 
grades 7-12 for risk categoiy 1. 



Grades 7*12 
At-Risfc Students 

Fall, End of Fall 1,021 
Fall, End of Year 1,021 
Spring, End of Spring 720 



Dropouts % 



tg3 
3tO 
127 



17.92 
30 J6 
17.64 



DATA SOURCES 



Data for the study were drawn from AISD files of at-risk stu- 
dents, dropout files from the end of each six-week period of the 
school year, and from a seven-year longitudinal dropout file 
maintained by ORE The longitudinal dropout file contains for 
each student the student's dropout status in each school year, as 
well as other basic information. The at-risk files contain for each 
at-risk student the reason(s) for being at risk as well as other 
basic information. 



9 



Note that the number of at-risk students changes from fall to 
spring. Thisdccline in enrollment occurs bwauscsomestudenis 
dropped out and others transferred out of the District during the 
fall. 

Dividing the number of dropouts during the spring by the spring 
cnrollmentyieldsarisk factor of I7.64%(127y720) to be used for 
those students enrolled in spring-only programs. Note that the 
17.64% for dropouts in the spring (one semester) compares to 
the 17.92% who dropped out In the fall (one semester). 
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nGURE4 
DROFOVTS AS FWCTION OF A T-USK STA TVS 
STUDENTS ENROLLED FALL 1999-90, 
GRADES 9.13 























END OF FALL SEMESTER 


ENDOF SCHOOL YEAR 


AS OF OCTOBER, 1990 












DropouU 




Dnopoats 


Risk Category 




Dropouts* 


%ofREsk 


Dropouu** 


%orRi&k 


Dropouts*** 




1^9-90 


Earalfmcnt 


1989-90 


Catcgofy 


1^-90 


Category 


1989-90 


Category 


1 


765 


155 


20.3 


249 


32.6 


247 


323 


2 


482 


7 


\3 


23 


4.8 


12 


Z5 


3 


27! 


3 


t.l 


12 


4.4 


16 


5.9 


4 


560 


9 


1.6 


66 


U.8 


62 


U.l 


5 


117 


4 


3.4 


9 


7.7 


7 


6.0 


6 


119 


2 


1.7 


9 


7.6 


10 


8.4 


7 


4 


0 


0.0 


0 


0.0 


0 


0,0 


8 


241 


g 


3.3 


12 


50 


12 


5.0 


9 


579 


u 


2.2 


19 


3.3 


12 


Zl 


10 


145 


18 


IZ4 


35 


24.1 


33 


22.8 


11 


387 


26 


6.7 


127 


32-8 


120 


31.0 


12 


245 


80 


32.7 


117 


47.8 


106 


43.3 


13 


232 


7 


3.0 


35 


15.1 


34 


14.7 


14 


1Z» 


24 


2.0 


76 


6.2 


74 


6.0 


15 


276 


5 


1-8 


43 


15.6 


36 


13.0 


16 


137 


9 


6.6 


33 


24.1 


32 


23.4 


17 


183 


39 


21-3 


70 


38.3 


67 


36.0 


18 


252 


23 


9.1 


95 


37.7 


89 


353 


19 


346 


12 


3J 


77 


22.3 


85 


24.6 


20 


364 


13 


3-6 


28 


7.7 


24 


6.6 


21 


291 


12 


4.1 


38 


13.1 


34 


11.7 


22 


363 


6 


1.7 


47 


13.0 


38 


10.5 


Tout 


7^7 


475 


6-3 


1,220 


16.1 


1.150 


15.2 


V 














. . J 



*Total fall high school dropouts » 678; therefore, 203 (29.9%) not identtHed as at risk. 
Not at-risk Dropouts/Not at-risk enroUment « Risk rate for not at risk students. 203/9JQ1 « 2.2. 

• •ToUl end of year high school dropouu - 1,800; therefore. 580 (32.8%) not identified as at ask 
Not at-risk Dmpouis/Nol at-risk enrollment « Risk rate fornot at-risk sttidcnts. 580^.634 ^ 6,0 

• • 'Total fall high school dropouts = 1 J48; therefore. 598 (34.2%) not identified as at risk. 

Not at-risk Dropouts/Not at-risk enrollment « Rtsk rale for not at-raik students. 598/9,i0l * 6.6 
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HGURES 

DROPOUTS AS FUNCnOS OF AT-RISK STATUS 
STVDBNTS SmOUED FALL I9S9.90, 
GRADES 7-8 



r 

• 




















END OF FAiX SEMESTER 


END OF SCHOOL YEAR 


AS OF OCTOBER. 1990 
















OrooouiA 


Risk Categoty 






% of Risk 


Dranotits** 


% of Risk 


DtX>DOUlS*** 


% of Risk 


1989-90 




fOfiQ on 








19^-90 


CatCflorv 


1 


25$ 


do 






4^ "CI 


60 


23-4 


2 


288 


3 


1.0 


5 


1.7 


12 


4^2 


3 


56 


A 


An 

v.v 


A 


Ir.v 






4 


N/A 














5 


127 


0 


00 


4 


3.2 


6 


47 


6 


138 


0 


0.0 


8 


5.8 


6 


4.4 


7 


N/A 














8 


259 


I 


0.4 


5 


!9 


11 


4.4 


9 


324 


I 


03 


5 


1.5 


7 


2,2 


10 


73 


6 


82 


18 


247 


16 


21.9 


11 


N/A 














12 


120 


25 


20.8 


50 


41.7 


46 


38-3 


13 


N/A 














14 


909 


7 


08 


27 


10 


41 


45 


15 


N/A 














16 


N/A 














17 


152 


7 


4.6 


28 


18.4 


20 


13.2 


18 


N/A 














19 


N/A 














20 


82 




1.2 




2.4 


4 


49 


21 


388 


5 


1.3 


17 


4-4 


19 


4-9 


22 


N/A 














Total 


3.172 


84 


2.6 


230 


7.3 


250 


7.9 


V 














J 



^Toiat fall middle/}un!Dr high school dropouis » 138; therefore, $4(39.1%) not identified as at risk. 
Not at-risk Drojx>uls/Not at-risk cnrolltncRt « Risk rate for not at risk students. 54/S.793 « 0.9. 

•*Total end of year middle/juntor high school dropouu = 409; therefore. 179 (43.8%) aot identified as at h*k. 
Not ai-ri&k Dropouts/Noi at-risk cnroflrBcnl » Risk rate for not at risk students. 179/6J95 = 19. 

•••Total fall fniddle^uninr high school dropouts » 517; therefore. 267 (51.6%) not edentifled as at nsk. 
Not at-risk Dropouts/Not at-nsk cnmltmcnt - Risk mte for no! at<r^ik students. 267/5.793 - 4.6 
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nGURE6 

DROPOUTS AS FUNCnON OF AT-RISK STAWS 
STVDESTS ENROLLED SPRINO 1 989-90, 
GRADES 9-1 2 

























END OF SPRING SEMESTER 


AS OF OCTOBER, 1990 












Dropouts 




[yropouts 










DropouU 






VP Ol KISK 






1989*90 


P nmltcn tf^nt 


1QBQ <VI 


Categoiy 




Category 






1 
* 






IT T 














t A 

Id 


3.0 














o 
¥ 




1 1 








A 
*9 




J f 




<1 
31 








C 


1117 


e 


4.7 




A "7 










/ 


0.3 




7 ^ 






7 




0 


DO 


0 


0.0 








Cm, 


4 


1.8 


9 


4.1 






o 




6 


l.l 


6 


1.1 






10 


107 


17 


15.9 


16 


15.0 






tl 


318 


101 


31,8 


88 


27.7 






12 


131 


37 


28.2 


40 


30.5 






13 


204 


28 


137 


25 


12.3 






14 


1,141 


52 


4.6 


52 


4.6 






15 


254 


38 


15.0 


30 


11.8 






16 


122 


24 


19.7 


23 


18.9 






n 


137 


31 


22,6 


27 


19.7 






18 


205 


72 


35.1 


64 


31.2 






19 


2^9 


65 


21.7 


68 


22.7 






20 


•35 


15 


4-5 


16 


4.8 






21 


251 


26 


10.4 


23 


9.2 






22 


331 


41 


12,4 


29 


8.8 




V 


Total 


6,560 


745 


11.4 


719 


11 0 


J 



♦Total spring high schoc?! dix^pouts * 1J22; iherefore, 377 (33 6%) not tdcnttfi^ as al risk- 
Not at-nsk Dropouts/Not at-risk enrollment = Risk rate for not at risk students. 377/8,955 = 4.2 

•♦Total spring high school dropouts « 1,120; therefore, 401(35.8%) not identified as at risk. 
Not at-risk Dropouis/Noi at rtsk enrollment « Risk rate for not at-risk students. 401/8.955 4.5 
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Compare those numbers with the 3036% of students who had 
dropped out as of the end of the sixth six weeks who wem in fail 
or year-long programs. The 17,64 will produce a more accurate 
prediction for dropouts from spring programs than would the 
30.36. 

As mentioned earlier, we are now calculating separate risk rates 
for spring and fa i by level (high school or middle school). These 
separate risk ates will provide more accurate predictions of 
dropouts and will provide t)etter comparison of programs to 
determine program effectiveness. 

What mt the results (tf the appikoiions used m 

See Attachment 1 for results of the application of th^ proce- 
dures to dropout prevention programs for 1989*90. TbJsspedCic 
cross-program romparison looksat dropout rates as of October, 
1990, for students served by dropout prevention programs in the 
fall of 19S9. Similar charts are available for students served in 
the fall as of the end of foil and as spring as of the end of spring 
and as of October. See Attachment 2 for results of the applica- 
tion of these procedures to the individual campuses. 

Comparisons of programs with each other at a specific point in 
time can be made by looking at charts similar to Attachment 1. 
If we wish to compare a program with itself across time or 
programs with each other at different points in time we need a 
chart like the one below. 





Suai^4t of Srmcslcsv 

1 2 


FoUoviBK October 








AlP 


FaU 


4?0 


76.4 


48.6 






Spring 


105.0 




^u3,6 




Mcfilcr (year long) 




37.0 


107.1 


99J 




Teenage Parent 


Fall 


0 


22L7 


232.4 






Spring 


465.7 




3793 




Gndcs9*i2 






ALC 


Fall 


94.9 


205.7 


139.0 






Spring 


217.1 




104.0 




Renaissance (year fang) 


S9.9 


1233 


106.5 




Zcnifh 


Fail 


87J 


90.5 


106.7 


J 




Spring 


69.6 




85.0 



From this type of chart it is possible to detect consistency of 
programs across time* to detect which programs are better in fall 
versus spring, and to detect from which programs or which 
semester of service students arc more likely to return to school 
in the fall of the following year 

With the realization that this type of chart* white useful* was not 
easy to read or interpret* we searched for an alternative presen- 
tation format. Aspin-off of this procedure has allowed a simple. 



Ftuzer ^ Ugon 

visual representation of the effectiveness of dropout programs. 
The dropout risk ratio is converts to + +. , 0* or - deter- 
mined by the direction and distance of the ratio from 100. See 
Attachment 3. 



f 
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99.S 


0 
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232.4 










PAL 










0 




I7U 




EBlDdhtiMflv: too (4- or 'to potato) « or ^ t00(*i>or- 


SOpoiniB) 


^ ^^9r*^ SecAtlaclioicfalL 




J 



IMPORTANCE OF LONCITVDtNAL DATA BASES 



The importance of well-maintained* longitudinal data bases for 
the evaluation of dropout prevention programs cannot be 
understated The decision by ORE in 1984 to create a longitu- 
dinal data bas^ to reflecv ue dropout status of all AISD secon- 
dary students has had an incalculable impact on local, and even 
state* policymaking. Alt discussions of the dropout issue ulti- 
mately revolve around the dropout rare* and without reliable 
data would be hypothetical* pointless* and a waste of time. 



EDUCATIONAL IMPORTANCE OF THE STUDY 



This new methodology: 

(1) Provides a statlstlral method for comparing the 
effectiveness of dropout prevention programs* 

(2) Uses evaluation criteria actually related to the 
event of dropout/graduation* 

(3) Adjusts for the level of risk of subjects in 
different pn^rams. 

The capability to Identify correctly which programs are success- 
ful with at-risk students is critical if dropout intervention efforts 
are to be effective. With reliable evaluation information* Dis- 
trict resources will be effectively targeted. Ineffective dropout 
prograrus will not continue to drain District and community 
resources. 
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ATTACHMENT I 

CROSS-PROGRAM COMPARISON 
PREDICTED AND OBTAINED 1989-90 DROPOUT RATES 

FALL, 1989 
AS OF OCTOBER, 1990 



Frazer & Ligon 



Grades 9-12 



PrograiD 



Group 

Dropout Risk 



Enrollment 



Predicted 
.Dropout 
# Rate 



Obtained 
Dropout 
# Rate 



Obtwnedas 
%of Predictctj 



ALC, Fall, 1989. Behavioral Sludnets 2.7342 

CIS, Fall. 1989. Grades 9-12 655.4 

CVAE, FaU, 19S9, Ondes 9-12 8462.1 

Evening Sdwol. FaU, 1989. Otadet 9-12 2,104.0 

ORAD. FaQ. 19S9, Refenal Cbde - t, Rev 2,751.4 

ORAD, Fail, 1989. Refetnl Code - 3, Rev 3.712.1 

GRAD. Fall, 1989, Refciral Code « 4, Rev 1.916.9 

JotmsUm CCP Computer Ub Fall, 1989-90 1,048.1 

Jonnston Dropout Reooray Program 269.0 

Mentor. FaU, 1989 1.341.4 

PEAK, FaUt. 1989 1.716.7 

Renaissance Progrua at Johnston 751 .4 

Robbins, Fall, 1989, Grades 9-12 3,503.8 

SenkMB Receiving PAL. Servkcs, Fall 1^ 1 .431.7 

Teenage Parent Center. FaU, 1^ 1,791.9 

Transitional Ac»temic Program 597.9 

Zenith. Fall. 1989. Grades 9-12 5,530.8 



140 
43 
421 
76 
ISI 
207 
106 
45 
23 
133 
113 
93 
188 
127 
111 
74 
210 



27 
7 

86 
21 
28 
37 
19 
10 
3 

13 
17 

8 
35 
14 
18 

6 
55 



19.5 
15.2 
203 
27.7 
18Ji 
17.9 

lai 

233 
11.7 
10.1 
15.2 

8.1 
186 
113 
I6.I 

8.1 
26.3 



TOTAL 



40,418.8 



2261 



404 



17.9 



38 
6 

lis 

36 
26 
36 
34 
12 
5 

12 
24 
8 
51 
12 
51 
15 
59 



27.1 
14.0 

2ao 

47.4 
17.2 
17.4 
32.1 
26.7 
21.7 

9.0 
21.2 

8.6 
27.1 

9.4 
45.9 
20.3 
28.1 



543 



24.0 



139.0 
91.5 
1373 
171.0 
94J 
97.0 
177.4 
114.5 
185.9 
89.5 
139.8 
106J 
145.6 
83.8 
284.6 
250.9 
106.7 



1343 



Grades 7-41 




Group 

Program Dropout Risk 


Enrollment 


Predicted 
Dropout 
# Rate 


Obtained 
Dn>pout 
# Rate 


Obtained as a 
% of Predicted 


Academic Incentive Program, Fall, 1989 617.2 
ALC Fall, 1989, Behavioral Studnets 7583 
ALC, Fall, 1^, Overage Students. Grade 1 .741 5 
as. Fall 19S9» Grades &S Z66.2 
CVAE, Fat!, 1989, Grade* 6^ 349.0 
Juniois Receiving PAL Services, Fan 1 ,294.0 
Mentor. FalM989 803.8 
Robbtns, Fall 1989. Grades 6-8 2.105. 1 
Teenage Parent Center. Fall 1989, Grade 172,1 
W!N» Fall, 1989. Grades 7 8 276.8 


56 
82 
74 
25 
65 

180 
99 

102 
10 
38 


6 11.0 
8 9.2 

17 23.5 
3 10.6 
3 5.4 

13 7.2 
8 8.1 

21 20.6 

2 17.2 

3 7.3 


3 5.4 
21 25.6 
38 51.4 

5 20.0 

4 6.2 
13 72 

8 8.1 
4? 41.2 
4 40.0 
7 18.4 


48.6 
276.9 
2182 
1878 
114.6 
100.5 

99.5 
199.5 
2324 
2529 


TOTAI. 

\ 


731 


84 11.5 


145 19.8 


17Z9 



15 



14 
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ATTACHMENT 2 
PREDICTED AND OBTAINED DROPOUT RATES 
AS OF END OFSBCTHSDC WEEES 1989-90 



Predicted Obtained 





Group 




Dropouts 


Dropouts 


Obtafned as • 




Dropout Risk 


enrol iwent 


f 


Rate 


# 


Rate 


% of Predicted 






10^ 


27 


16.5 


60 


36.0 


999 ft 

222.0 


Ancterson 






120 


8.4 


97 


6.8 


o0*6 


Austin 




499C 


171 


9*6 


193 


tK2 


1i6«4 


Bowie 




22Jo 


AAA 

190 


8*5 


75 


3.4 


t A A 


crocKett 




4ft7n 


192 


10. S 


137 


7.4 


94 A 

71 .9 


jonnscon 


22146.5 


4TCe 


221 


12p6 


301 


17.3 


137.1 


1 B ft 




4V74 

i371 


ito 


8.0 


61 


4.5 


56.1 




invu.% 


4XftA 

1004 


172 


10.7 


224 


14.1 


131 .6 


RCvftI liHI 




4/ 

14£> 


14S 


10.2 


126 


9.0 


88.2 




1S4S7.8 


1429 


155 


10.8 


170 


12.0 


110.9 


koddt 




cOT 


37 


17.5 


56 


27.5 


156.8 


Travis 


161S0.4 


1436 


162 


10.9 


128 


8.7 


80.0 


S4i>total 


17D207.0 


16751 


1702 


10.2 


1633 


9.8 


96.4 




2921.S 


137 


29 


21.3 


48 


46.2 


216.6 


Austin Stat 


364.4 


43 


4 


8.5 


7 


16.3 


192.3 


ucwiopacnc 


m «u 


72 


7 


13.7 


2 


4.0 


29.3 


K10 ufanw 


flftT ft 
CHI>.U 


AC 
4» 


8 


17.8 


i3 


29.5 


165.3 


noMcwouno 


iS5. 1 


22 


2 


8.3 


• 

1 


4.5 


C/ 4 

54.1 


Ifary Loc 


tOX ft 


24 


2 


9.3 


10 


41.7 


447.2 


PI < ^<»MM 

Li t Tvon 


2w4. 7 


five 
100 


26 


24.6 


4 


3.8 


t5.5 


TBOTn^a Kar 




ItS 


32 


16*4 


85 


43.4 


265.0 




lie. J 


15 


1 


7.5 


*> 
2 


13.3 


177.6 


Uni larcn we 


▲ ft 

6.0 


4 

1 


0 


6.0 


0 


0*0 


A A 

0.0 


dUDCocai 


ft 


0«^ 


112 


17.4 


172 


28.4 


4 AT / 

163.4 




10l^fT»0 


4TXOA 

1 f oy*> 


1614 


10.4 


# one 

1805 


« A C 

10.5 


4ftft T 

100.7 


AlC 


2823.S 


223 


9A 


49 *9 


or 


39-9 


315.1 


oBui cneK 


f 


Aft9 


25 


3.7 


i6 


2.3 


X4 O 

01 .y 






QOO 


9A 
cO 


t O 


9A 

20 


3.y 


4ftft 9 








Tft 

30 


3.2 


3 


A 9 

0.3 


O X 

Y 




ft 


All 


99 


3.7 


94 

21 


3.4 


09 O 

y2.y 


rUllfKirC 


>9ftT 1 

CCOf m I 


\ 


9* 

23 


3.7 


4a 

It 


3.1 


OT 4 

63*1 


KCB » r 


XiXtiK ft 


ftftT 


Tft 

30 


3.7 


4T 

13 


1.6 


AX ft 

43 «y 




9n&^ n 

CU^7.U 




9A 

20 


3.8 


4C 


2.8 


#4.3 


nBr»» in 


^ry.o 


f to 


35 


4.5 


33 


4.2 


AV 0% 

93.9 






oo% 


24 


3.6 


9 


1.3 


36.3 


Plarchison 


3199.1 


727 


32 


4.4 


7 


1.0 


22.7 


w. ifcnry 


4*Fft V T 


470 


4T 

1 f 


J^. 7 


o 


4 T 

1 . / 


45.8 


rCBl b« 






24 


4.2 


30 


5.2 


1C9 .9 


Porter 


2736.6 


749 


27 


3.7 


21 


2.8 


76.6 


Robbins 


2S98.4 


130 


26 


20.0 


62 


47.7 


238.6 


Std»totai 


39107.0 


9173 


391 


4.3 


372 


4.1 


96.2 


Austin Stat 


217.4 


48 


2 


4.5 


7 


14.6 


322.4 


Deve(opiaent 


29.0 


10 


0 


2.9 


0 


0.0 


0.0 


Rio Grande 


447.5 


46 


4 


9.7 


4 


8.7 


89.4 


Konetound 


1G6.7 


9 


1 


11.9 


3 


33.3 


280.9 


Nary lee 


69.6 


23 


1 


3.0 


5 


21.7 


717.1 


Teenase Par 


309.0 


32 


3 


9.7 


8 


25.0 


258.9 


Shoal Creek 


69.6 


24 


1 


2.9 


5 


20.8 


717.2 


Children Ce 


8.7 


3 


0 


2.9 


0 


0.0 


0.0 


Subtotal 


1257.5 


195 


13 


6.4 


32 


16.4 


254.3 


Total 


40364.5 


9368 


404 


4.3 


404 


4.3 


99.8 
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ATTACHMENT 3 
EFFECTIVENESS OF DROPOUT PROGRAMS 



PROGRAM 


5 

£22i 










"'OUTCOHES 


SPRING 








Dropout^ 


Attendance 0 « scl ol i ne 




6PA 


OroDOMta 


Attendance Discipline 


Fs 




Grades 9-12 




















AlC Beh. 


$ss 


I 




0 


0 


♦ 


0 




0 


♦ 


CCP (Johnston) 


s$ 


L 






0 


0 


0 


0 0 


0 


0 


CIS 


s 


E 


0 




0 




0 




0 




CVA£ 


ss 


L 




♦ ♦ 


0 


0 


0 




0 




00 Rec. (Jan '90) 


s 


0 




0 


♦ 












Evening School 


$ss 


L 


















GRA0*1 Cln School) 


$$s 




0 


0 0 


0 


0 


0 


0 


0 


0 


GRAP^Z (S'd/E'd) 


m 


£ 






0 


0 








0 


GRAD-3 (Dropouts) 


$$s 


E 


0 




0 


♦ 






0 


0 


GRAD-4 (Truant) 


s$$ 


E 




0 


0 


0 






0 


Q 


Nentor 


t 


L 


♦ 


0 ♦ 


0 






♦ 


0 




PAL 


% 


E 




0 ♦ 


0 


0 




0 


0 


0 


PEAK 


$s 


L 




0 


0 


♦ 






0 
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Cost 

(1 to 3 SJ : 1 = Lo« 
3 ° High 



'Funding 

E s External 
L Local 
0 a None 



ERIC 



Program fnpact: 

Positive » ♦ Strongly positive » 
Negative = - Strongly negative » 

No Change « 0 A Blank = Insufficient nurbers or Does not apply 
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Fratzer & Ligon 
(S OR r?) 



ALC 8eh. « Alternative Lefirning Center students with behavioral problems S 

CCP (Johnston) « Cofqputer coofietencfes progrm at Johnston High S 

CIS « CoosQunities in Schools S 

CVA£ s Coordinated Vocational Academic Ectoation S 

DO Rec. « Dropout Recovery in January '90 at Johnston Hi^ S 

Evening School « Evening school • . • • S 

6RAD-1 s In school students who the intervention specialists served (grant Research About Dropouts) . S 

GRA0*2 « Suspended or expelled students who the intervention specialists contacted S 

GRAO-3 « Students who were already dropouts who the intervention specialists worked with • . . • S 

GRA0*4 8 Students t^o were in school but absent 9-29 days who the intervention specialists served S 

Nentor « Kentor T 

P/l ^ Peer Assistance Leadership S 

PEAK » Practical, Effective, Appropriate Knowledge . S 

Renaissance - Math* reading- language arts block of 9th gractors at Johnston Y 

Rofabins « Alternative school S 

TAP » Transitional Academic Program S 

Teen Parent Center » Campus for pregnant girls S 

Touch s counseling and tutoring at Crockett S 

Zenith * individualized curriculum contracts $ 

GRADES 7*8 

A!P ^ Academic Incentive Program S 

ALC Behp 3 Alternative Learning Center students with behavioral problems S 

ALC Overage = ALC students were sent there because of their age S 

CIS ^ ConiTKjnities in Schools S 

CVAE » Coordinated Vocational Academic Education S 

HSSI > Hispanic student scholarship initiative where UT students tutored average achievers .... S 

Nentor = Kentor Y 

PAL c Peer Assistance Leadership S 

Robbins = Alternative school S 

Teen Parent Center » Campus for pregnant girls S 

WIN « Work incentive program for borderl ine-of -fai I ing students ^ 

DECISION RULES 

All areas of impact are calculated during the semester in question with two exceptions. Attendance and discipline at the 



ALC are measured after the semester, and all areas of impact for Dropout Recovery are measured for the spring of 1990. 

Differentials 

Attendance: 2 points or 5 or more points or 
Discipline: 2 points [♦ or -1; 5 or more points or 
Fs : 1 point [♦ or -J; 2 or more points or — 

CPA : 2 points [♦ or -]; 5 or more points or 

Dropouts :10 points [♦ or -1;50 or more points or — 

17 

o Hi 

ERIC ^ 



A program is categorized as year- long if 60% or more 
of the same students were enrolled in it for both 
semesters. 



